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ABSTRACT 

NCCOSC RDT&E Division (NRaD) is conducting research into ways of improving decision support 
systems (DSS) that are used in tactical Navy decision making situations. The research has focused on the 
incorporation of findings about naturalistic decision-making processes into the design of the DSS. As part of that 
research, two computer tools have been developed that model the two primary naturalistic decision-making strategies 
used by Navy experts in tactical settings. Current work is exploring how best to incorporate the information 
produced by those tools into an existing simulation of current Navy decision support systems. This work has 
implications for any applications involving the need to make decisions under time pressure, based on incomplete or 
ambiguous data. 



BACKGROUND 

Our research at NRaD is part of the TADMUS (TActical Decision Making Under Stress) project, funded by 
the Office of Naval Research. That project is generally involved with looking into new ways to enhance the decision 
making of Navy personnel in tactical command and control situations. 

The particular focus of TADMUS is on the area of anti-air warfare (AAW), and involves situations in which 
shipboard commanders must make decisions about the nature and intent of aircraft that are in the vicinity of the ship. 
Because these situations can involve jet aircraft armed with missiles, decisions must sometimes be made in seconds 
even though the available information can be incomplete or ambiguous. The decisions are made by six person 
teams, where the ultimate responsibility for making decisions lies with the ship’s commander (CO) who is closely 
aided by a tactical action officer (TAO). 

The TADMUS research has taken two primary directions. Part of the work is involved with looking at new 
training approaches for the decision teams. The other primary effort involves the investigation of new approaches in 
the area of decision support systems. We are involved with the DSS area at NRaD. 

A large part of the early DSS related work under TADMUS involved the investigation of the decision- 
making strategies used by experienced Navy personnel in actual tactical situations. That research took the approach 
of looking into naturalistic decision making. Naturalistic decision making emphasizes gathering data about how 
experienced decision makers make their decisions in real world settings. This approach is to be distinguished from 
the more artificial approaches often used in decision research, where inexperienced subjects are tested in doing 
unfamiliar tasks in artificial settings. One of the general ideas emerging from studies of naturalistic decision making 
is that it appears that experienced human decision makers are much better at making good decisions than is often 
suggested by more traditional research approaches. 

The early TADMUS research identified two primary naturalistic decision-making strategies being used by 
experienced Navy personnel [5]. In most situations, those decision makers use a strategy referred to as recognition- 
primed decision making (RPD). This strategy relies on the use of prior experience to suggest how prototypical 
patterns of data may be sufficiently close to currently observed data to guide decisions about the current data [6]. 

When prior experience proves insufficient to guide current decisions, a second strategy comes into play using 
explanation-based decision making, or story generation. This strategy involves the construction of a few alternative 
explanations that account for the current data, followed by an evaluation of which explanation is the most plausible 
[8]. In both the RPD and story generation situations, selection of a course of action is expected to be automatically 
generated as a consequence of the situation assessment. 

Two tools have been developed under TADMUS to model those strategies. An RPD template based tool 
takes the approach of matching current data items against a set of stored templates [7]. Those templates represent a 
set of typical scenarios that can be expected to occur in given tactical settings. The tool estimates the degree of fit 
between actual current data and the stored templates, and brings templates to the user’s attention when there is a 
sufficiently good fit. A second tool called SABER (Situation Assessment By Explanation based Reasoning) 
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constructs alternative explanations that suggest how all current data can be accounted for in reaching distinct different 
conclusions [2,3]. SABER makes use of a few heuristics to evaluate which of the alternatives is most plausible 
Also in connection with the TADMUS DSS work, a simulation facility has been set up that models 
existing Navy shipboard systems. A number of experiments have been run at that facility to develop empirical 
evidence about what kinds of problems arise [4], A new set of experiments will begin shortly to compare 
performances with and without the use of the new tools. All experiments are done with experienced Navy personnel. 

NEW IDEAS IN DECISION SUPPORT 


New Information Products 

One of the key hypotheses relied on in developing the two decision support tools is that the use of models 
of people’s cognitive strategies will promote the ability of the tools to enhance people’s performance with those 
strategies. At a general level, the idea is to match the internal models of the tools to the naturalistic decision- 
making strategies of the users, as a means of achieving a cognitive fit between the way both the tool and the user 
approach the problem. That fit is expected to lead to improved performance by facilitating a more rapid and accurate 
use of the strategies. The desirability of achieving a cognitive fit has been emphasized in recent work in the human- 
computer interface field dealing particularly with tools intended to enhance human performance [10]. 

Development of these tools has led to an ability to produce new kinds of information products for the 
decision makers. Some of the key new information products available from the RPD tool are: scenario events 
shown in a timeline, predicted events, and suggested responses. These products arise directly out of the tool’s use of 
timeline based templates representing event scenarios. 

The more important new products produced by the SABER tool are: an ability to group data according to 
the hypotheses each piece of data directly supports, the related ability to indicate where important pieces of data are 
missing, and a review of possible ways to account for contradictory pieces of data. This kind of information arises 
from the tool’s ability to construct alternative explanatory structures to account for arbitrary sets of data. 

Some of these new information products are illustrated in Figures 1 and 2, where virtually all of the parts of 
those figures represent either completely new kinds of information or a new way of presenting the information, as 
compared to existing Navy decision support systems. These figures show information related to a given aircraft 
track, where the tracks are typically shown as symbols on a geographic display. Figure 1 illustrates an early 
proposed interface for the new tools. There are nine separate windows illustrated which in counterclockwise order 
show the following: (1) a three dimensional display of the track history for a selected track; (2) a list of events of 
interest for the track, keyed into the 3D display; (3) a list of other information related to the track; (4) a bar graph 
indicating the most likely hypothesis as to the nature of the track; (5) a template presentation showing the key 
elements of the hostile intent hypothesis; (6) a geographical display showing various tracks, with the currently 
selected track highlighted; (8) a list of suggested responses with regard to the track; and, (9) a more general list of 
alerts, which bring items of possible interest to the user’s attention. 

Figure 2 illustrates a way of presenting some of the information available through the SABER tool. The 
window to the right shows the grouping of data items according to four high level possible conclusions. Also 
shown is the ability to indicate important pieces of data that may be missing, where in that window such items are 
greyed out. The bottom part of the window to the left shows where assumptions can be used to explain away some 
pieces of data. The upper portion of that window represents a way of indicating which of the four conclusions is the 
most plausible according to the SABER tool. 

Building the Interface 


A major goal in designing the interface is to try to satisfy the partially conflicting goals of providing new 
kinds of information, while at the same time minimizing the time required for users to assimilate the available 
information. The need for minimizing user interaction time results directly from the real time nature of the tactical 
situation. As a result of these goals, we are directing part of the design effort at determining what new information 
is most critical to the decision maker and at finding ways to compress the presentation of that information. 

A related problem lies in presenting the information produced by the system in a way that does not suggest 
to the user that the actual decisions are being made by the system. It is important to note that since these tactical 
situations necessarily involve incomplete and ambiguous information, it is not possible for any computer tool to 
generate completely reliable conclusions. In that kind of situation it seems preferable to leave the ultimate decision 
up to the user, particularly where the decisions can have life or death consequences. One of the principal questions 
that needs to be resolved in this connection is whether or not the system should indicate to the user the system’s 
own evaluation of the situation. 
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Figure 2. SABER related information products. 


In arriving at some final design ideas, we have radically changed our approach to integrating the new 
information products into the existing DSS framework. The RPD and SABER tools were originally developed as 
highly interactive, standalone tools. At that time it was expected that the tools would be incorporated into the 
overall DSS system through separate CRT displays. However, it became clear from both the real time nature of the 
problem and the identity of the users that that approach was not viable. In fact, it appeared that if that approach were 
adhered to we would have reached the situation of needing to add a new person to the decision team who could 

mediate between the two tools and the other team members. 

The real time constraint leads to a general need to avoid the need for user interactions with the system. In 
addition, as the TADMUS empirical work progressed, the decision was made to focus on the two members of the 
decision making team who have the ultimate decision making responsibility: the CO and TAO. Those individuals 
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are officers who typically do not lake a very active role in interacting directly with the DSS, making it even more 
desirable not to introduce new interactive features. 

T r C L. nC i^ Ult W3S t0 t * ec '^ c on a sm g |c CRT display, that would combine the most critical information 
products of the RPD and SABER tools. The general idea of the intended product is illustrated in Figure I It should 
be noted that the actual display will be in color rather than the black and white shown in Figure I . The main point 
ere is that Figure I shows that a variety of information can be incorporated in a display on a single monitor 

Also of importance in Figure I is the point that the only kind of user interaction called for is to click on a 
track ol interest in the geographical display window. Related information of interest is then automatically changed 
in the other windows. This action is one that is already required on existing DSS’s. It should further be pointed out 
that the symbology used in the geographical display window shown in Figure I is unchanged from standard Navy 
usage, except that color has been added to our product. 

In determining how best to fit together the RPD and SABER tools we have developed the view that the 
tools tit together in a two level approach. At the more specific level, the RPD tool is able to determine how well 
current data fits into one or more predefined template representations of typical scenarios. Where there is a good fit, 
the RPD tool can bring templates to the user’s attention along with accompanying information products The 
SABER tool is seen as performing a more general role that is useful when current data may not fit any of the 
predef ined templates well. Viewed at this level, the role of SABER is seen as supplying tentative, alternative ways 
of explaining the data in terms of a limited number of high level possible conclusions. 

Critics 


In part, our interface design ef torts are now being guided by principles related to the use of critics in user 
interlaces. Critics are devices that have received increasing interest in the user interface community in recent years 
[1,9]. Critics have been implemented in a number ol forms, but the essential idea is to have a built-in mechanism 
lor making suggestions to the user about how to proceed with a given task. This kind of mechanism is particularly 
useful where the computer and user are thought of as working together to cooperatively solve a problem of some 
sort. Critics have been most often used in cooperative design settings. 

That approach fits in well with what we want to accomplish with the TADMUS DSS research. The goal is 
to achieve a situation in which decisions are made partly as a result of cooperative problem-solving. What is then 
needed is to identify which activities are done best by computers and which by people in this setting, with the view 
that by integrating those activities the overall cooperative performance will exceed what either the computer or the 
person could do alone. 

One way of utilizing the critic concept would be to build a model of the user into the interface such that the 
system could calculate from the user’s actions what the user was trying to do at any given time. Based on that kind 
of knowledge the critic could automatically supply new information as it reasoned that the user needed it. Although 
we believe that approach is worth further study, at the present time we are not pursuing it. The problems are that 
the approach does not appear to be appropriate for real time applications, and in our situations it is likely that users 
would object to having the system decide for them when to change displays and what to display. 

Instead, we have adopted the approach of trying to critique the user’s decision-making processes in a way 
that is purely passive and transparent to the user. Essentially, we try to emphasize through our display that there are 
always alternative ways of explaining a given set of data. While doing that we show some possible ways in which 
the existing data fits together, and indicate where the fact that some pieces of data are missing may be important. 

This approach is illustrated in Figure 2 where the user is shown how a current set of data relates to the top level 
conclusions about whether an aircraft may be friendly or not. That figure indicates that the interface can compactly 
show alternative ways of accounting for data as long as there are a limited number of top level conclusions. 

POSSIBLE COMMERCIAL APPLICATIONS 

There are applications for this research in any setting involving the need for real time decision making. 

Some examples are: some kinds of medical settings, weather forecasting, and possibly air traffic control. These are 
situations in which decisions must sometimes be made quickly, and may need to be based on incomplete or 
ambiguous data. 

The specific RPD and SABER tools developed for the TADMUS project are not likely to be of value in 
other settings, since they have both been specially tailored for the AAW command and control setting. However, it 
is reasonable to assume that the decision-making strategies modeled through those tools are the same strategies used 
by other decision makers in stressful, real time decision making areas. Where those strategies are used, it should be 
straightforward to create tools that use the basic underlying approaches of the two tools. The part that would not be 
so easy, would be tailoring the interface to meet the needs of new domains. 
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There are a few key lessons learned in our project that should be kept in mind in developing new special 
purpose tools based on these strategies. The most important lesson is that there are essentially two approaches to 
making new tools like this available to users. One way is to build in a lot of functionality along with interactive 
features, with the assumption that either the user will spend a significant amount of time learning to use the tools or 
that a new human assistant will be used to mediate between the user and the tool. The other option is to try to 
simplify the interface as much as possible, and require as little interaction as possible. We have chosen the second 
approach at NRaD. A related lesson is that it is vitally important to identify the ultimate intended user as early as 
possible so that realistic approaches can be taken with regard to the use of features that require extensive interaction. 

CONCLUSION 

Initial research done under the TADMUS project has produced a theory about the strategies used by expert 
Navy decision makers in tactical command and control situations. Two computer tools have been developed that are 
tied into that theory: the SABER tool and an RPD tool. Work is now being done to determine how best to present 
the information produced by those tools to users. The expectation is that such information will lead to improved 
decision making. 

The principles being developed through this work have application in any areas involving the need for real 
time decision support systems. The most obvious area of applicability is in medicine where emergency situations 
can raise problems with time constraints, incomplete data, and ambiguous data. Those are exactly the kind of 

problems that are the focus of the TADMUS efforts. 

Our design process has suggested a few design principles of particular interest in these real time settings. 

At a general level it can be noted that simply using models of user cognitive strategies may not directly suggest 
good ways to construct the interface. We believe that the models do lead to producing the kind of information that is 
needed, but that determining how to present the information remains a hard problem. More specifically, we have 
found that although direct manipulation is normally viewed as a desirable approach for letting users manipulate data, 
in this real time setting it is desirable to minimize such manipulations in order to minimize the time the user must 
spend dealing with the interface. In addition, we are using graphical data presentations where possible, but not to the 
extent that users are required to learn an extensive new set of symbols. 

At a general level we have focused our design efforts on the idea that the RPD and SABER tools tit 
naturally together through a two level model of the kinds of information that need to be presented to users. The 
RPD tool can supply information related to specific, detailed representations of scenarios, while the SABER tool 
relates data to more general possible conclusions. In addition, we are exploiting the idea that our overall interface 
can be viewed as a type of critic. 
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ABSTRACT 

This paper presents an overview of a Decision Support System (DSS) that incorporates 
Knowledge-Based (KB) and commercial off the shelf (COTS) technology components- 
Knowledge-Based Logistics Planning SheU (KBLPS) is a state-of-the-art DSS with an interac.tve 
map-oriented graphics user interface and powerful underlying planning algorithms. KELPS h 
been designed and implemented to support skilled Army logisticians to P^P™ ^ahtate 
logistics dans rapidly, in order to support corps-level battle scenarios. KBLPS represents a 
substantial advance in graphical interactive planning tools, with the inclusion of intelligent planning 
algorithms that provide a powerful adjunct to the planning skills of commodity distribution 

planners. 


INTRODUCTION 

The complexity and dynamics of commodity distribution planning - both in commercial and 
milkary domains - require decision support tools that provide much more than data base access and 

spreadsheet solutions This paper presents an overview of a D f e f c ^ n ^ 

incorporates Knowledge-Based (KB) and commercial off the shelf (COTS) technology 

components. 

The Knowledge-Based Logistics Planning Shell (KBLPS) is a state-of-the-art DSS with a rich 
mao-oriented fnteractive graphics user interface and powerful planning algorithms that has been 
designed and implemented to support skilled Army logisticians to much more rapidly and 

evaluate logistics plans to support corps-level battle scenarios than currently possible. These plans 
mav be developed as contingencies against future possible scenarios or in direct support of troops 
JiteS it ether casl the abilhy to build, evaluate, and improve plans in a fraction of the 
time now possible has been a major objective which is in the process of being met. 

KBLPS is an appropriate blend of Artificial Intelligence, Knowledge-Based, conventional, and 
commercial of/the shelf technologies that, taken together, provide a logistician with * powerful 
tool to help define, analyze, and evaluate very complex planning problems quickly. The 
Logistician can configure the problem/scenario with a through-the-screen object-oriented approac , 
as well as give guidance to the algorithm by (optionally) setting a number of parameters, the 
algorithm constructs distribution plans, involving significant computational complexity on beha 
oAhe logistician. As a consequence, the logistician can spend more time analyzing and assessing 
plans than in generating them. 
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